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Abstract 

This paper proposes a framework for analyzing program design features that seem to matter in 
implementation. The framework is based on findings from a study conducted by the Consortium 
for Policy Research in Education (CPRE) between 2004 and 2007 that explored how reform 
ideas and practices created by five external provider organizations were enacted in a national 
sample of high schools. Eour interrelated design factors are examined: emphasis, level of 
complexity, approaches to teacher and administrator engagement, and availability of 
implementation supports. This framework offers a guide for local educators, provider 
organizations, and researchers to generate questions and collaborate to strengthen 
implementation and realize program goals. 


Introduction 

Implementing deep and long lasting change in schools is hard. Too often, changes 
envisioned do not fully reach hoped for outcomes. Adaptations may reflect only the surface 
attributes of an innovation, lacking the deeper intent upon which the reform is based. We know 
from practice and decades of research on implementation that even the most thoroughly studied 
and elegantly crafted program is likely to be altered as it is brought to life in a particular 
classroom, school, or district. While some alterations strengthen the viability of a program in a 
specific setting, others may seriously undermine a program’s ability to facilitate desired change. 
The challenge is to be able to recognize and anticipate both. 

We need a framework to help program providers and local education leaders identify key 
components of designs that matter in implementation. Such a framework can help providers and 
education leaders make decisions about the appropriateness of the design for a particular school 
setting and predict potential trouble spots in the implementation. Once the decision to 
implement the design has been made, a framework can help educators and providers make 
predictions about where the design is likely to encounter challenges and plan accordingly. 

Increasingly districts are partnering with external organizations to assist in improving 
core facets of schooling — instruction, learning, and school community. In 2004, Millot 
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estimated that school improvement purchases were climbing rapidly, from $50 million in 1996 to 
$1.4 billion in 2001. Since then, that figure has dramatically increased as government 
accountability requirements have placed greater pressure on school systems to improve academic 
outcomes for all students and offered financial incentives to turn to outside organizations to 
provide support. 

Given the significant investment, much attention has been directed toward improving the 
implementation and scale-up of externally created educational reforms. Cobum (2003) identified 
four dimensions of scaling-up educational reforms: depth, spread, sustainability, and ownership. 
Attention to these four dimensions is also useful for understanding implementation within a 
particular school or school system. Depth of use involves moving “beyond the surface stmctures 
or procedures to alter teachers’ beliefs, norms of social interactions, and pedagogical principles 
as enacted in the curriculum” (p. 2). Spread involves both the number of people or schools in 
contact with the reform. There is an important interaction between these first two. Program 
developers want the reform ideas and activities that are spread across a faculty to have depth 
rather than be mere superficial changes. Sustainability involves the nature of the reform’s 
persistence over time. Ownership focuses on the extent to which “authority and knowledge of 
the reform” (p. 7) moves from the external provider to those individuals, schools and school 
systems using the design. 

School improvement efforts developed by external provider organizations pose a unique 
set of implementation questions. Unlike a school system’s homegrown improvement efforts, the 
basic program blueprints are designed by the provider and intended for implementation in 
multiple school locations. The creators of the program often have a strong sense of what needs 
to happen in order for the reform to meet the desired outcomes based on beliefs about the nature 
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of change needed and how to effect change. Those beliefs may reflect years of research and 
work in school systems. At the same time, local educators have a deep knowledge of their 
particular context, needs, and priorities. And it is there — in that district, school, or classroom — 
that educators and the provider negotiate how the blueprint will come to life. In the hallmark 
Rand Change Agent study, Berman and McLaughlin (1978) describe this push-pull between an 
externally-developed initiatives and the setting as a process of mutual adaptation in which both 
the program and those who are engaged in the implementation are changed. Datnow, Hubbard, 
and Mehan (2002) argue that we have vastly “underestimated the co-constructed nature of 
implementation” (p. 10). This has significant implications for understanding why an externally 
created blueprint that depicts the program’s implementation as a logical set of components and 
sequential steps can look quite different in subtle and substantial ways when it is enacted in a 
particular school. 

Program providers have employed a variety of approaches to address this core tension 
between maintaining fidelity to a program design and adapting to meet the needs and priorities of 
the implementing school or district. These approaches reflect the designers’ theories of how to 
effect change in schools. Some designs are accompanied by highly detailed guidance. Providers 
argue that specificity is necessary to remain faithful to the design’s core principles in 
implementation and then achieve the desired ends (Ball & Cohen, 2003; Berends, Bodilly, & 
Kirby, 2002; Shiffman, Massell, Goldwasser, & Anderson, 2006). Detailed instructions of the 
program’s components, materials, and processes leave little ambiguity for those tasked with 
enacting the program. Thus, educators implementing the program have concrete guidance as 
they begin putting the program into practice. The provider has greater confidence that the 
program will achieve desired results, that those results can be evaluated, and that the program 
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can be replicated. Other providers opt for a looser blueprint that fosters deep involvement from 
the educators implementing the design. These providers argue this co-construction is essential to 
gamering educator buy-in and making the program viable in that setting (Datnow, et. ah, 2002; 
McLaughlin & Mitra, 2002). Without educator buy-in — particularly when a design calls for 
difficult behavioral changes — commitment to the reform in the face of competing priorities is 
likely to be weak and educators will not have a sense of ownership of the reform ideas and 
practices. To address these competing pressures, Thompson and Wiliam (2007) — among 
others — argue for a ‘tight-but-loose’ approach to implementation that favors “obsessive 
adherence to central design principles” (p. 2) as well as space for addressing the needs, 
resources, and conditions of a particular school as long as these do not conflict with the ‘tight 
part.’ 

An additional complication lies in recognizing that the providers’ designs are never static 
(Berends, et. ah, 2002; Datnow, et. ah, 2002; Shiffman, Riggan, Massell, Goldwasser, & 
Anderson, 2008). Providers continually revise and refine designs with deepening knowledge of 
implementation in schools and changes facing an organization from the policy environment and 
the market. Thus, a program as designed in 2006 is likely to look different in subtle or 
substantial ways in 2015. 

At the outset, both education leaders and program developers need tools to assess 
whether a program is a good fit. Hatch (2002) observes that to make an informed selection, 
education leaders need a deep understanding of the array of externally created programs and 
what might be required of a school or district in terms of time, resources, and amenability to 
change. He argues that too often education leaders do not have the time and resources to fully 
vet programs. A poor fit or underestimated implementation drains scarce resources, educator 
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time and energy, and ultimately does not serve students well. Once implementation begins, local 
education leaders need to be able to anticipate where and when alterations are likely, predict the 
impact of those alterations on the program’s ability to facilitate the desired outcomes, and plan 
accordingly. 

This paper offers one framework for analyzing program design features that seem to 
matter in implementation — a design’s emphasis, complexity, engagement, and support. The 
framework is based on findings from a study conducted by the Consortium for Policy Research 
in Education (CPRE) between 2004 and 2007 that explored how reform ideas and practices 
created by five very different external providers were enacted in a national sample of high 
schools. Eollowing a brief summary of the study and the program designs, this paper will 
discuss the framework and each of the four design features that influenced implementation in the 
study high schools. 

The Study 

Eor two academic years a team of CPRE researchers explored the ways in which five 
instructional improvement programs created by external provider organizations were enacted in 
15 high schools across the United States. The study was funded by the U.S. Department of 
Education’s Institute for Education Sciences (lES).^ Two primary research questions guided the 
study: 

• How do the reform ideas and practices of external change agents interact with school 
environments and teacher attributes to change instructional and organizational practices 
in high schools? 

1 Funding for this study was provided by the U.S. Department of Education’s Institute of Education Sciences (Grant 
#R308A960003). 
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• What factors both within and outside high schools explain differing levels of 
understanding and enactment of programs? 

(Supovitz & Weinbaum, 2008, p. 10) 

Not only did these five programs have different foci, each took different approaches to 
implementation. Some programs provided highly specified instructions for rolling out the 
program; other designs emphasized co-construction of the reform from the beginning. 
Description of Programs Included in the Study 

The programs included in the study represented three common types of instructional 
improvement programs found in high schools at the time: whole school reforms, targeted literacy 
initiatives, and data use strategies (Gross & Goertz, 2005). Whole school or comprehensive 
school reform programs (represented in this study by First Things First and High Schools That 
Work) focus on making school- wide changes rather than concentrating on one area of a school or 
school population. These programs typically focus on improving instruction, school 
organization, and relationships in a coherent and comprehensive manner (Vernez, Karam, 
Mariano, & Di Martini, 2006). These models have grown in prevalence with federal and private 
funding incentives over the past couple of decades. The Education Commission of the States 
(2015) estimates that over 8,000 schools are using comprehensive school reform models. 
Targeted literacy initiatives (represented in this study by Ramp-Up to Advanced Literacy and the 
Penn Literacy Network) address a widespread and enduring concern that adolescent students do 
not have the literacy skills needed to complete grade-level coursework (Biancarosa & Snow, 
2004). In 2013, 64 percent of eighth grade students scored at basic or below basic on the 
National Assessment of Educational Progress (U.S. Department of Education, 2014). In contrast 
to a whole school reform, targeted designs such as those examined in this study focus on the 
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needs of a specific subset of students and teachers needing additional support. Data use 
strategies (represented in this study by SchoolNet) focus on the use of data to improve 
instruction and student performance. The standards movement, accompanying accountability 
requirements, and significant technological innovation prompted the current emphasis on 
collecting, reporting, and using student performance data. Yet collecting, managing, analyzing, 
and using data to inform decisions remains a complex process that requires extensive 
cooperation and preparation (Lachat & Smith, 2005; Massell, 2001; Supovitz & Klein, 2003; 
Shiffman, et. ah, 2006; Wayman, Stringfield, & Yakimowski, 2004). Table 1 provides a brief 
summary of the five designs included in the CPRE study. Note that these designs are not static. 
Table 1 presents key characteristics of the designs as they existed at the time of the study. 


Table 1. Programs Included in CPRE Study^ 


Whole School Reforms 

First Things First 

Created by the nonprofit Institute for Research and Reform in Education (IRRE) in 1989, this program 
seeks to strengthen relationships among students, teachers, and families; improve instruction that 
prepares students for college and careers; and reallocate resources to support these goals. 

At the time of the study, the design called for reorganizing students and faculty into theme -based small 
learning communities, instituting a family engagement system, and providing teacher professional 
development. The implementation plan was detailed and occurred over multiple years. Leader and 
staff positions were designated to guide the reform. Teacher buy-in for the reform was expected to 
come in completing program activities. 


High Schools That Work 

Created by the Southern Regional Education Board (SREW), this program seeks to improve student 
achievement by fostering a school culture of high expectations and ongoing improvement. To do that, 
the design emphasizes enhancing instructional capacity and collegial discussion, and moving staff 
through a strategic planning process. 

At the time of the study, the design called for the school community to vote to accept the reform in the 
school and then participated in a strategic planning process focused on increasing achievement for all 
students. The design materials included a set of key practices that constituted an effective, high 
quality school as defined by the provider. These practices addressed what students should leam, 


2 These program descriptions reflect the designs as they existed during the study period. 
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conditions for optimal learning, and responsibilities of teachers. A technical assistance visit helped 
staff and other stakeholders evaluate their strategic plan. Each school was assigned an SREB 
representative to make technical assistance visits and provide ongoing, tailored support. HSTW 
schools also had access to the organizations network of high schools across the country. 


Targeted Literacy Initiatives 

Ramp-Up to Literacy 

Created by the National Center for Education and the Economy (NCEE), this design is an unbundled 
component of NCEE’s whole school reform — America’s Choice. The Ramp-Up program seeks to 
bring middle and high school students performing one to two years below grade level up to grade level 
within two years. To do that, the program employs a workshop model of instruction that scaffolds 
learning and is organized around specified routines and rituals. 

At the time of the study, the program provided a highly detailed comprehensive curriculum for two 
English courses. Supervisors and teachers assigned to Ramp-Up followed the design’s detailed 
curriculum, pacing calendar, classroom arrangement, and student evaluation plan. The design called 
for structural changes for the two courses including block schedules, class size limits, and looping — in 
which the teacher moves with the students to the second year of instruction. The school was required 
to purchase a classroom library as well. The design also called for cross-age tutoring in which Ramp- 
Up students read to younger children to foster student motivation. Some professional development 
from NCEE was included as part of the basic package. 


Penn Literacy Network 

Created by professors affiliated with the University of Pennsylvania, the basic design follows a 
professional development model for individual teachers. The program goals include offering teachers 
and school leaders a framework for improving literacy instruction, learning, and assessment; providing 
opportunities for reflection and professional community; and working to build a research-based 
curriculum. 

At the time of the study, the design engaged teachers and schools through voluntary participation in 
workshops and courses. The design did not specify instructions regarding how to put program ideas 
into place. The design focused on facilitating teacher change through reflective practice and enhanced 
instructional literacy skills. This design did not call for changes to the school structure; rather change 
was concentrated in individual teacher practice in a classroom. 


Data Use Strategy 

SchoolNet 

This design was developed by the SchoolNet company in 1998 to assist districts with collecting, 
managing, and making data accessible to teachers, administrators, and the school community. 

Through tracking instruction and student performance, analysis, and reporting results, the design 
assumes teachers and administrators will adjust curriculum and instruction. Student academic 
achievement is expected to improve with greater attention to individual students’ learning needs. The 
design also envisions an online professional community to share lesson plans, instructional strategies 
with colleagues. 

At the time of the study, SchoolNet offered multiple internet -based products and technical assistance 
for purchase. SchoolNet worked with school districts to select products and then develop an 
individualized district data system and supports. As a district-led effort, the design did not follow a 
particular approach to student learning or implementation plan. The basic design relied on a train-the- 
trainer model although additional technical assistance from the company was available for purchase. 
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Data Collection and Analysis 

To identify the 15 study high schools, CPRE researchers asked leaders from each of the 
five provider organizations to identify three high schools that showed promise implementing the 
program. The focus of the study was on early implementation years. Thus, each provider 
organization identified two high schools that were in the early stages of implementing the 
reform. The provider representatives were also asked to identify a third high school designated 
as the “mature” site. The mature site was selected as an example of where the program had been 
in place for at least three years and was considered to be operating well. CPRE researchers 
conducted multiple site visits to early implementation high schools over the two-year data 
collection period and conducted one site visit to each of the ‘mature’ high school sites. The 15 
study high schools were diverse by region, school and district size, and student demographics. 

At all 15 high schools, the researchers interviewed teachers and school leaders and administered 
a survey of teacher attitudes about the program. Over 500 interviews were conducted at the 
school and district level. In addition, 25 interviews were conducted with senior staff at the five 
provider organizations. 

Interestingly, the variability found in the study schools was not necessarily attributable to 
the length of the time the reform had been in a particular school. In some cases, early 
implementing schools were operating the program more consistently with the blueprint than the 
mature sites. To explain the implementation experiences in the 15 schools and the variability 
across schools, CPRE researchers focused on four key areas: program design, school social 
networks, school leadership, and school district role. 

^ A full account of the research study is available in Supovitz, J.A. & Weinbaum, E. H. (Eds.) (2008). The 
Implementation Gap: Understanding Reform in High Schools. New York: Teachers College Press. 
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The framework discussed in this paper focuses on program design. Data that informs this 
paper are drawn from interviews with provider staff, teachers, and school and district 
administrators; and materials produced by or about the five instructional improvement programs. 
We first analyzed the designs as they were presented by the provider. Next, we examined the 
designs as they were implemented from the perspective of the provider, school, and district staff. 
We reviewed interview transcripts that described modifications to design ideas and strategies. 
Finally, we analyzed the data across the schools and designs to identify patterns in the 
implementation in these 15 high schools. 

Design Factors Framework 

A framework can direct the attention of providers and education leaders to dimensions of 
the design that are likely to be more or less susceptible to modification. This can be of 
particular benefit to schools and districts with limited time and resources, aiding selection of an 
appropriate design to meet desired goals (Hatch, 2002). This framework can also assist 
providers and local educators to plan implementation together. 

This framework conceptualizes the process of implementing a particular design as a 
sequence of interactions between the provider staff and the implementing school or district staff. 
The resulting ‘enacted’ program in a particular school or district, thus, is a product of that 
ongoing dialogue and negotiation. In the CPRE study, the nature of those provider- school 
conversations evolved over time. In particular, the provider organizations were more apt to 
negotiate design components once implementation was underway, rather than prior to 
implementation (Shiffman, et. ah, 2006). 

The design factors framework directs our attention to four key design factors and two 
crosscutting themes that help to explain the implementation variation found. The four design 
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factors include emphasis, complexity, engagement, and support. The crosscutting themes center 
on the level of detail provided to enact the design and the interactions among design components 
that matter in implementation. Figure 1 provides a graphic illustration of the framework. The 
four design factors are depicted. The broken arrows represent the interrelatedness of the factors. 


Figure 1. Design Factors Framework 
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Design Emphasis 

Designers have to make ehoiees about what to emphasize and when (Barnes, Khorsheed, 
de Los Rios, & Correnti, 2006; Berends, et. ah, 2002; Desimone, 2002; Kronley & Handley, 
2003). Inevitably there is a hierarehy among design eomponents and that prioritization is 
eritieal to implementation (Bodilly, Keltner, Purnell, Reichardt, & Sehuyler, 1998). During 
implementation, prioritized design eomponents are more likely to reeeive greater attention from 
a provider, be aceompanied by more instructions and support, and more fully enacted than design 
components receiving less emphasis. The hierarchy among design components may be implicit 
and only fully apparent when challenged. 

Analyzing a design’s theory of action — essentially how a program is supposed to be 
rolled out and achieve program goals — ^provides key information about components emphasized 
by developers during implementation. Two themes embedded in the theory of action help to 
illuminate design emphasis. The first theme centers on a designer’s belief about where the bulk 
of the program’s development should occur, i.e., at the provider organization or in the local 
school or district (Desimone, 2002; Glennan, Bodilly, Galegher, & Kerr, 2004; Kronley & 
Handley, 2003). As discussed earlier in the paper, decisions in this area reflect philosophical 
beliefs about ownership of the reform and importance of the local context. These decisions also 
reflect providers’ strategies for generating buy-in among educators tasked with implementation. 
A second theme embedded in a design’s theory of action centers on the role of organizational 
conditions in achieving program goals. Does the design rely on particular structural 
arrangements such as scheduling, grouping students and faculty, and distributing authority? 

These choices reflect providers’ beliefs about how individual practice is related to a school’s 
organizational conditions; how to most efficiently put the program in operation; and how to 
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demonstrate impact in a local school environment in which attention to this program competes 
with many other priorities. 

Two key dimensions of design emphasis are: 1) centrality of the component to the overall 
program goals, and 2) the implementation sequence in which components are introduced. 

Centrality 

Design components that are central to a program and its underlying philosophy tend to be 
the components most likely to be accompanied by greater resources, specificity in instructions, 
and pressure from the provider to implement. In their study of comprehensive school reforms, 
Vemez and his colleagues (2004) identified two types of design components — those components 
that are central to a design’s identity, and those that take a supporting role in realizing central 
components. In the CPRE study, we found a similar pattern. For example, the Ramp Up design 
had a highly detailed plan for classroom instruction but left the supporting cross-age tutoring 
component to the schools to figure out. This component was either not implemented or weakly 
implemented by the study schools. Consistent with the ‘tight-but-loose’ approach discussed by 
Thompson and Wiliam (2007), we found that providers in the CPRE study held tight to 
components that were central to the program’s identity and were less willing to negotiate 
adaptations in these areas. When a First Things First school proposed organizing students in 
small learning communities based on student ability, the provider strongly objected and 
responded by intensifying support including providing additional resources and staff to 
implement the theme-based, equitable small learning communities. 

Central components of a design can conflict during implementation. Often, ensuing 
provider-school system negotiations may reveal an implicit hierarchy among design principles. 

A particular location of conflict lies in a designer’s beliefs about program goals and commitment 
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to co-constructing a program with local educators. Of course, whenever educators are consulted 
for their knowledge and expertise, a designer raises the risk that these educators may reject or 
devalue a particular component (Shiffman, 2012). Implementing the High Schools That Work 
design in the CPRE study schools provides an illustration. Specifically, this model seeks to 
foster high expectations for all students. Reaching this goal is achieved through a plan 
developed by the school community. When provider staff perceived these two were in conflict, 
we observed that the provider staff prioritized high expectations for students at the expense of 
faculty co-construction. 

Given that prioritization is inevitable, it is valuable for both provider staff and education 
leaders to take a close look at the proximity of individual program components to central ideas 
and goals of the design. Such an examination can bring clarity regarding what the provider 
believes is important in the design and implementation process. This can assist providers and 
local education leaders to identify components more or less likely to be accompanied by the 
provider’s elaboration and support. With less provider attention, a local school or district may 
need to step in to fully enact the component. The relative centrality of a component also has 
implications for the nature of provider-implementer negotiations around program modifications. 
School or district efforts to alter central design components are likely to meet with greater 
resistance from provider staff. 

Sequence 

A second dimension of design emphasis concerns the order in which design components 
are introduced during implementation. There is evidence from multiple studies that components 
introduced during the early implementation years are those components most likely to take hold 
and be fully enacted (Barnes, et. ah, 2006; Vemez, et. ah, 2006; Shiffman, et. ah, 2008). 
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Instructional improvement programs are enacted in dynamic local school environments where 
educators are contending with multiple priorities, funding streams, changing student needs, and 
policy mandates. There are windows of opportunity in which the leadership, resources, and 
attention are present. Those school and district leaders who selected and approved the design are 
typically still present and most invested in the design achieving the desired results. With time, 
schools and districts will move on to other pressing needs, priorities, and opportunities for 
change. Leader and educator turnover can also dilute both understanding and commitment to a 
reform. 

In the CPRE study, those design elements introduced early in implementation tended to 
be more fully realized than components introduced later. At the beginning, the provider and 
local education leaders and staff devoted the most attention and resources to the program. These 
early components also tended to be accompanied by the most detailed instructions for 
implementation. For example, the Ramp Up to Literacy design prioritizes specific instructional 
approaches supported with detailed guidelines. These were the program components that were 
readily apparent in the study schools during the first year. The High Schools That Work 
provider staff and schools invested heavily in defining the problem and reaching consensus on 
plans to address the problem. A school’s strategic plan was thus in place before moving on to 
the next design activities in the study schools. 

When a design calls for structural changes as a central feature of the design — those 
components are more likely to be introduced early. Structural arrangements may need to be in 
place before subsequent activities and processes can occur. However, having structures in place 
does not ensure subsequent changes will automatically follow — particularly when those changes 
are also complex. In the CPRE study, the First Things First and school staffs invested significant 
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attention, time, and resources to create small learning communities intended to foster supportive 
conditions for improved instruction. However, the difficult instructional changes that flow from 
the small learning community arrangement require additional and sustained support at a point 
when implementers’ attention and energy can wane. Across the provider-identified ‘mature’ 
sites, we found schools and districts shifting priorities and attention to new problems and new 
funding priorities. Thus what was in place remained, while later components were abandoned or 
more loosely followed. 

It is important to evaluate designs in terms of what elements will be put in place early. 
Those early components may be the only ones that take hold before local educators turn to other 
priorities — or move on to other jobs. This dimension of emphasis prompts questions for 
providers and local leaders: Are the components we most desire to see in place? Will those 
components get us far enough along towards our goals — and satisfy the very reasons the design 
was selected in the first place? 

Complexity 

The second set of design factors fall under the umbrella of complexity. Here complexity 
refers to the level of difficulty associated with bringing the reform ideas into practice (Fullan, 
2007; Mazmanian & Sabatier, 1989; Rogers, 2003). Rogers (2003) describes the complexity of 
an innovation in terms of the ability to understand and use, arguing that there is a negative 
relationship between the perceived complexity of an innovation and the rate of adoption. Fullan 
(2007) elaborates further that the degree of complexity depends on the difficulty, skill required, 
amount of adjustments in beliefs, instructional approaches, and curricular materials needed. The 
framework includes two dimensions of complexity. The first concerns the level of abstraction 
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that accompanies the concepts and practices teachers, leaders, and staff are called upon to 
implement. The second concerns the level of technical difficulty. 

Abstraction Level 

At a superficial level, it is relatively easy to reach consensus around oft-touted goals of 
school improvement initiatives. Who is not an advocate of more student engagement, high 
academic expectations, and a supportive school community? However, as individual educators, 
leaders, and providers begin to drill down to determine what high expectations look like in 
practice — and the steps to get there — consensus breaks down. 

Providers embed abstract program goals into their designs with varying degrees of 
elaboration and instructions regarding how to bring them to reality. The design must define and 
operationalize concepts that often defy easy operationalization. As noted previously, some 
designs structure defining those concepts and what they mean for practice as a process of co- 
construction between the provider and the local educators (McLaughlin & Mitra, 2002). Such 
co-construction takes time and variation is assured. Other designs attempt to encode those 
concepts into specific practices and organizational conditions that are believed to facilitate 
bringing an abstract concept to life. For example, restructuring a school into small learning 
communities is intended to engender engagement and professional community. The challenge 
becomes whether a structural change does in fact facilitate the intended goal or remains simply a 
scaffold without substantive changes in relationships. 

Abstract concepts that can be particularly difficult to translate into practice include ‘high 
expectations,’ ‘professional community,’ and ‘student engagement.’ These concepts can be 
embedded into more concrete design components such as changes to school structure, policies, 
or activities. But, these concepts also involve unique beliefs and behaviors of individual teachers 
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and leaders. For example, there is substantial room for individual interpretation regarding what 
‘high expectations’ should look like in a classroom. A design might embed high expectations in 
concrete, measurable changes such as encouraging students to take higher-level coursework. 
While more students may take high-level classes, those courses can be watered down based on 
teachers’ beliefs about student readiness for this work in their classrooms. In the same vein, 
professional community may be facilitated by structural conditions such as common planning 
time — though the extent to which that professional community is fully realized and supports the 
larger program goals may depend on the group of individuals participating in the community. 
Likewise, student engagement may be strengthened through the use of specific instructional 
practices or student groupings — though without teachers’ and leaders’ concerted efforts to foster 
engagement in their routine interactions, these efforts may remain but procedural artifacts. 

Fullan (2007) posits that while the level of complexity can pose challenges to 
implementing schools and districts, greater change can be possible when more is attempted. The 
lesson here is to pay particular attention to the level of abstraction accompanying a particular 
design concept, the ways in which the concept is operationalized, and the additional work that 
may be needed to fully realize the concept in a particular school or district. As Mazmanian and 
Sabatier (1983) argue, even difficult problems can be addressed with adequate understanding of 
the challenges posed and supports needed. 

Technical Difficulty 

A second type of complexity concerns the technical capacity required of individual 
teachers, leaders, schools, and school systems. Even when design components are fully 
articulated and accompanied by detailed instructions, appropriate local education resources and 
conditions may not be present. A school or school system will face technical difficulties if it 
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does not possess the existing infrastructure or staff knowledge and skills to implement design 
components with fidelity. 

The basic structures and operations of school are often stubbornly resistant to long lasting 
change (Murphy & Adams, 1998; Tyack & Cuban, 1995). A design’s structural changes that 
require coordinating schedules can be particularly challenging to implement and sustain. For 
example, scheduling and sustaining a block schedule for selected classes but not the entire school 
can pose an administrative challenge that reappears over time as new initiatives arrive. Thus, a 
central structural component of design is intermittently at risk of erosion. 

The volume of data — and technology to access that data — have exploded over the last 
quarter century. If harnessed, student performance data can provide a powerful tool for assessing 
and monitoring student progress, informing subsequent instruction, and communicating with 
stakeholders about student and school performance. However, introducing new technology often 
poses enormous challenges for a school system’s infrastructure and the skills of data managers 
and users. In order for teachers to access current data, the data needs to be up to date and 
available; and teachers must to know how to navigate such a system. Subsequently using that 
data to inform instructional calls on another set of skills that often require additional support. 

For these reasons, the districts implementing SchoolNet in the CPRE study faced significant 
challenges building a district-wide data system and utilizing the full capacity of that system. 

For local education leaders selecting a design, it is important to evaluate the level of 
technical difficulty posed to a school or school system. Those designs that pose a high level of 
technical difficulty will require significant investments of local time and resources to be fully 
implemented. High levels of provider and educator leader attention are often devoted to the 
initial investment of resources, yet most design components require ongoing maintenance. For 
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those design components that pose technical difficulties, education leaders and providers need to 
ask whether a school or district will be able to devote the needed staff, funds, and commitment to 
sustain a design into the future in the face of competing priorities. 

Engagement: Teachers and Administrators 

Gamering staff commitment to a program that involves complex changes in instruction, 
learning, and school community is critical for deep implementation and sustained use. A core 
preoccupation in the organizational change and leadership literatures concerns fostering staff 
commitment. Leader and staff commitment can be understood as the ideal outcome of a process 
that begins with awareness and engagement. 

School improvement designs approach school-level educator engagement in a variety of 
ways. Some designs call for voluntary participation. This approach presumes that those teachers 
and administrators choosing to participate bring with them an interest and willingness to engage 
in the program. However, this approach can be slow and have limited reach into a school’s 
faculty. Teachers and administrators opting not to participate may never fully understand what 
the design — its components and processes — mean for practice. For other school improvement 
designs involvement is not optional. Teachers and administrators are required to undertake the 
implementation work. In doing so, the argument goes, that staff will deepen understanding of 
and participation in the initiative. Ideally, staff commitment to the reform will follow (Connell, 
2002 ). 

Three common dynamics helps to explain the level of engagement observed in the CPRE 
study. The first dynamic centers on teacher and administrator perceptions about the relevance of 
the design to their work. The second dynamic centers on the extent to which teachers and 
administrators perceive the design addresses a problem they consider important. Finally, teacher 
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and administrator engagement is enhanced when they see a positive impact on a recognized 
problem. 

Relevance to Educator Work 

Teachers and leaders pay attention when a program impacts their daily work. That initial 
awareness and participation can either support the program or work against it. For example, 
whole school reforms require changes that can impact virtually every aspect of a teacher’s 
professional life. Whether resisting or embracing the changes, they are aware and involved — 
even if the level of engagement is superficial. In contrast, a design that targets a subset of 
teachers and students is less likely to receive much attention from those who are not directly 
impacted. 

Proximity to a program’s core work tends to deepen educator understanding of a program 
and investment in its successful implementation. In particular, individuals involved in 
constructing a design from the beginning may receive more training directly from the provider 
and be actively engaged in the adaptive work needed to implement the design in their school, 
classroom, or district. In the CPRE study, these individuals articulated the greatest 
understanding of the design and program goals, and expressed deeper commitment. The 
implementation experiences of the SchoolNet districts illustrate this. As a design initiated by 
district leaders that involves extensive collaboration and technical work between the provider 
and the district office to create and operate a data system, the central office staff developed a 
deeper understanding and commitment to the initiative. Teachers, particularly when using the 
SchoolNet system was optional, described little to no use of the system and limited awareness of 
what the system could do. 
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Engaging formal leaders is critical to implementation. When school and district leaders 
play key implementation roles, they are more apt to secure and protect needed supports and 
participate in aligning a design to school goals. In the CPRE study, we observed variation in 
instructional leadership around the reforms. When a design called for formal leaders to play an 
instructional role, these leaders expressed a deeper understanding and investment in the design. 

This dynamic directs provider and local education leader attention to ways in which a 
program will or will not impact teachers’ work within a school or school system. The teachers 
and leaders most directly impacted will be those who are most likely to pay attention to, reflect 
on, and engage with program ideas. They may be instrumental in co-constructing a program and 
spreading ideas to colleagues. They are likely to be those who can most effectively advocate for 
and protect the program. As such, both providers and education leaders need to think 
strategically about which individuals are likely to be selected to implement a program and what 
they are likely to need to succeed. 

Alignment 

The second dynamic concerns alignment between provider and educator perceptions of 
problems facing a school and the range of viable solutions. When teachers and administrators 
believe the design addresses a problem they consider to be important, they are more likely to 
take interest in the design. Conversely, when a design addresses a problem that is misaligned 
with teacher or leader priorities, additional work is likely needed to foster engagement. Eow 
student engagement provides an illustration. Teachers face this fundamental problem in their 
daily classroom work with students. As such, teachers in the CPRE study often expressed strong 
interest in those design components that targeted strengthening student engagement. There was 
less need for discussion between provider staff and teachers about the value of addressing this 
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problem. Given the widespread teacher interest in improving student engagement, some designs 
deliberately introduce student engagement strategies early in the implementation to encourage 
teacher support for the program (Connell, 2002). By contrast, when teachers do not consider the 
problem addressed by the program to be a problem they will likely be less interested in the 
program and its efforts to address the problem. For example, while a literacy initiative may seek 
to improve student literacy throughout a school, teachers in subject areas other than language arts 
may not view literacy needs as their problem. In this scenario, more work will be needed to 
align teacher and provider conceptions of a problem as a means of deepening educator 
engagement. 

How a problem is defined and framed also shapes perceptions about the range of possible 
solutions. Thus, it is important to consider whether teachers and leaders recognize an issue as a 
problem to be addressed, and then agree with the design’s framing of the problem and solutions. 
In cases where there is misalignment, providers and local leaders overseeing implementation 
may need to plan additional work to reach a shared understanding of problems and potential 
solutions. 

Perceived Effectiveness 

The third dynamic concerns teacher and administrator opportunities to witness a 
program’s positive impacts. Observable evidence of success is likely to deepen engagement and 
further interest in a program. These opportunities are critical for motivating educators in the 
midst of uncertainty about what the program does and their capacity to carry it out in the context 
of competing obligations and demands. Some designs strategically introduce and structure 
design components to show evidence of success fairly quickly (Connell, 2002). The First Things 
First design calls for initiating simple strategies to foster student engagement in classes early in 
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implementation. In the CPRE study, teachers were able to quickly see the design’s positive 
impact on an issue they cared about — student engagement. Similarly, by following the Ramp Up 
plan for instruction, the teachers and education leaders were able to witness increased interest in 
reading among their students fairly quickly. They viewed this as a positive indicator of the 
program’s impact. 

This dynamic directs provider and education leader attention to look strategically at the 
nature of evidence available about a program’s impact for those who are engaged in putting its 
ideas into practice. What aspects of a program are likely to show positive impact? Will that 
impact be recognized? By whom? And, when is a positive impact likely to be evident? For 
those contending with the hard work and uncertainty of putting new ideas and practices into 
operation, experiencing a few positive impacts during implementation boosts morale. Long 
term, the ability to observe positive impacts has implications for deepening educator 
commitment to a program’s ideas, practices, and continued implementation. 

Support and Monitoring 

The fourth design factor that contributes to variation in implementation involves the 
availability and nature of supports and monitoring. Implementation is a dynamic and context- 
specific process. Access to timely and appropriate supports and monitoring can make or break a 
program’s implementation. Promising structural and content dimensions of implementation 
supports include those that provide frequent opportunities for ongoing and shared learning 
grounded in classroom practice and those that allow for ownership of reform by school actors 
(Berends, et. ah, 2002; Bodilly, 2001; Cohen, Peurach, Glazer, Gates, & Goldin, 2014; Garet, 
Porter, Desimone, Birman, & Yoon, 2001; McLaughlin & Mitra, 2002; Slavin, 2004). Yet, many 
of these supports are likely to incur high costs (Barnes, et. ah, 2006; Vernez, et. ah, 2006). 
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Furthermore, it is often difficult to anticipate the total cost of fully implementing a design at the 
outset. Both the providers and local education leaders feel pressure to contain costs. 

This design factor highlights three characteristics of implementation supports associated 
with program understanding and use on the part of local educators — access to provider staff and 
materials; communication between the provider organization and those who are putting the 
design into practice; and availability of supplemental supports when needed. 

Access to Provider Staff and Materials 

To make the kinds of substantive changes to teaching, learning, and school culture 
outlined in program designs often requires some form of professional development for those 
responsible for putting the changes into practice. While evidence of what makes professional 
development effective remains somewhat suggestive, approaches that include sustained 
engagement, the use of active teacher and provider learning, and collective participation show 
promise (Cohen, et. ah, 2014; Caret, et. ah, 2001; Wayne, 2008). Professional development can 
be extraordinarily costly. Program designers and local educators have to make difficult choices 
about the ways in which implementation supports are provided to contain costs, while also 
optimizing possibilities for implementing the kinds of deep, widespread, and sustained changes 
that are owned by school and district staff (Coburn, 2003). 

Professional development in support of program implementation can take multiple forms 
that range in both cost and intensity. One form is direct training and advising from the provider 
organization staff. Another is a train-the-trainer approach in which selected local staff receives 
more intensive training from the provider and then return to teach their colleagues. A third 
method is to provide educators with detailed prepared guidelines to review while implementing 
design components. Often, a design will employ a combination of these approaches. Given the 
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significant pressure on the part of the school systems and the provider to contain their own costs 
as it relates to professional development, providers often offer more intensive trainings but do so 
as an additional cost to the school system. School systems will attempt to minimize costs where 
possible, and professional development — given the potential expense and uncertain scope — is a 
likely candidate. 

Direct access to technical assistance from a provider has implications for deep 
understanding and use of the program ideas. The CPRE study found that individuals with more 
direct access to provider staff for technical assistance articulated a better understanding of the 
design and its components. In contrast, we found greater confusion among those individuals 
receiving training indirectly from their colleagues about the design components, and later more 
difficulty incorporating reform ideas and practices into their work. 

Timing is another factor. Teachers and leaders introduced to a design in the early stages 
may acquire a deeper understanding of a program and its goals. In the early stages of 
implementation, school, district, and provider resources and attention tend to be most 
concentrated. Thus, those receiving training in the early stages have access to more resources to 
learn about and collectively work through how to incorporate the reform into their work in the 
school and make it their own. Of course turnover is inevitable. As new teachers are introduced 
to program ideas later in the implementation, their training is more likely to be further removed 
from that initial intense implementation work. 

Provider guidance is also critical to prepare school and district leaders for their role in 
implementation. When leaders have a clear understanding of their role in a design, they are more 
likely to be committed to implementing the design and aligning resources. In the CPRE study, 
some of the designs offered fairly explicit instructions and training for leaders related to roles 
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and responsibilities as pertained to the program. Other designs offered little to no guidance for 
formal leaders about their role. 

Providers and education leaders face a difficult task in determining the nature of 
implementation supports. They seek a balance between containing costs and prospects for deep 
understanding and use of program ideas over time. This decision involves determining the level 
of access to provider staff and resources needed. Questions to guide this process include: What 
do teachers and leaders need in terms of knowledge and skills to successfully understand and 
realize the goals of the program — i.e., how complex are the ideas and practices being 
implemented? Will provider staff and implementing teachers and leaders have a shared 
understanding of these ideas and practices? If the answer is no, what level of access to provider 
supports will ensure that an idea or practice is implemented and sustained in a manner that 
supports program goals? 

Communication 

As stated previously, implementation is a dynamic process that is impossible to 
completely predict. As such, ongoing communication between a provider’s staff and those 
implementing the program is critical for clarifying a design, addressing challenges to 
implementation, and deepening understanding and use of the program in practice to meet the 
goals of both the school and provider. Provider staff brings deep knowledge about a program, 
how design components and the implementation plan support program goals, common stumbling 
blocks during implementation, and availability and uses of resources and strategies to support 
implementation. However, provider staff is likely to have varying degrees of familiarity with the 
specific implementation circumstances in a particular school. Provider staff may never directly 
communicate with those individuals who are implementing and using the program. When 
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communication between provider and school staffs is limited, there is more room for 
modifications that reflect the understandings and solutions of the school staff. Without deep 
knowledge and experience with design, school leaders and staff may not be able to distinguish 
between superficial and substantive adoption of program ideas. In the CPRE study, some 
designs included minimal opportunities for provider staff to visit the schools to observe the 
design in practice and offer technieal assistance. As such, provider staffs had limited awareness 
about what was actually happening in the study schools. 

When provider staff has limited contaet with those individuals involved in implementing 
designs, an important communication pathway is curtailed. The provider staff — those who are 
most familiar with the design, implementation experiences, and range of supports and 
challenges — cannot respond to the specific questions arising in a school or district as they 
emerge. A teacher’s question may be altered as the question travels through layers of translation 
or never even reach the provider. As a result, provider staff may never be able to identify micro- 
level misunderstandings and design modifications that ultimately undermine program goals. 

Supplemental Supports 

It is very difficult to anticipate the full range of supports needed to deeply embed an 
external design into classroom practice and school culture. Coupled with the cost pressures of 
provider-led training and other activities, school systems face constant pressure to minimize 
costs. However, providing additional supports — at the time they are most needed — is often 
critical to deep implementation of design components. Supplemental supports can involve 
professional development tailored to address specific problems, technical assistance visits, 
mentoring, and call-in support. As a principle, supports that require provider staff time tend to 
be expensive. Berends and his colleagues (2002) have argued that this is a significant challenge 
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for provider organizations. Providers must successfully market the need for supplemental 
supports to districts that are often reluctant to devote scarce resources further. In the CPRE 
study, provider and local education leaders readily agreed that additional supports were key to 
substantively implementing the design in ways likely to achieve program goals. Yet they all 
faced pressure to make cost-saving choices. When supplemental supports and monitoring were 
not available, teachers and leaders were left to make do, drawing on whatever school and district 
resources were made available. Where local support and monitoring were insufficient, prospects 
for deep and sustained used of program ideas were vulnerable. 

Crosscutting Themes 

Two themes cut across the four design factors. The first concerns the level of specificity 
that accompanies a design or design component. The second concerns the interactions among 
design factors. Both are important to consider in evaluating a design and prospects for 
implementation. 

Influences of Design Specificity on Local Interpretation 

Providers and implementing schools systems struggle to find an optimal balance between 
implementing the design with fidelity and allowing schools and districts the flexibility to adapt a 
program so it remains viable. As a whole, the five designs in the CPRE study took quite 
different approaches to providing detailed instructions for implementation. This same pattern 
was true when looking at individual components of each design. Essentially, some program 
components were accompanied by more detailed instructions than others, even among those 
designs with an overall greater degree of specificity. Program components with less guidance 
called for greater interpretation on the part of the school or district staff to put the component 
into place. As implemented in the 15 schools, these less specified components ran the gamut — 
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from virtual nonexistence, to a superficial piece of the program, to a deeply understood and 
rooted component that supported a program’s ability to achieve desired goals in that particular 
school or district. 

As a simple guideline when assessing the possibilities for implementing a design in a 
particular setting, providers and implementing school systems can examine the amount of detail 
that accompanies design components. Those components with less specific instructions are 
likely to require more interpretation on the part of individual educators and leaders in the school 
and district. Whenever more school and district interpretation is called for, there is likely to be 
greater variability and potentially less certainty of expected outcomes. Paying attention to the 
level of specificity that accompanies individual program components can help education leaders 
plan for the time, capacity, and resources needed to do this interpretative work. 

Interaction Among Design Components 

Most — if not all — designs seeking substantive changes in instruction, learning, and 
school culture comprise multiple components. It is the combination of components that are 
intended to help the school or district reach desired goals. The relationship between components 
is important to recognize and understand. Glennan and his colleagues (2004) argued that a 
change to one design component can impact the range of “decisions and responses available” (p. 
31) to implement other design components. In the CPRE study, this interaction occurred most 
often between key emphases of the design and efforts to engage teachers and administrators who 
were tasked with implementing the program. Desimone (2002) noted this tension, observing 
that even when providers strongly valued co-construction of the reform, implementing teachers 
called for specific instructions regarding how to put the design into practice. 
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Conclusion 

Selecting an externally created design that targets core aspects of schooling such as 
instruction, learning, and relationships is a significant investment for a school system and the 
provider. Too often providers, schools, and researchers are disappointed when a program does 
not reach its goals. Characteristics of the designs play an important role in channeling the 
interaction among the design, the provider staff, and those charged with enacting the design in a 
particular school or district. 

The framework outlined in this paper identifies four key design factors that can influence 
dialogue and provider-school/district negotiations as a design moves from blueprint to a living 
set of beliefs and practices. These four factors — a design’s emphasis, level of complexity, 
approaches to teacher and administrator engagement, and availability of implementation 
supports — function in concert with one another. Therefore, a modification in one area may alter 
the range of options and responses in another area. Similarly, the level of specificity that 
accompanies a design component carries implications for the amount of interpretation required 
of implementing educators, and variability in program enactment. 

This paper offers a potential framework for providers and local education leaders to more 
effectively and efficiently assess the appropriateness of a design for a particular school or district 
prior to making a selection. Once a program is selected, this framework can offer providers and 
busy local educators important clues about where implementation challenges and program 
modifications are likely to occur and thus make plans to provide critical supports. A framework 
focused on the program’s design, such as the one proposed here can generate targeted questions 
in these two areas. In the preceding pages, several questions have been offered to guide provider 
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and educator planning. This is by no means an exhaustive list, but rather a starting point for 
providers, researchers, and educators to explore how to work together to realize often elusive 
goals for students, and those who serve them. 
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